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Abstract 

A spatial instrument is defined as a spatial display which has been either geometrically or sym- 
bolically enhanced to enable a user to accomplish a particular task. Research we have conducted over the past 
several years on 3D spatial instruments has shown that perspective displays, even when viewed from the cor- 
rect viewpoint, are subject to systematic viewer biases. These biases interfere with correct spatial judgements of 
the presented pictorial information. The design of spatial instruments may not only require the introduction of 
compensatory distortions to remove the naturally occurring biases but also may significantly benefit from the 
introduction of artificial distortions which enhance performance. These image manipulations, however, can 
cause a loss of visual-vestibular coordination and induce motion sickness. Consequently, the design of head - 
mounted spatial instruments will require an understanding of the tolerable limits of visual-vestibular discord. 

Introduction 

The introduction of relatively low cost, interactive, high performance 3D computer graphics 
work-stations such as the Personal IRIS or the Megatek 928, and the certain prospect for further minia- 
turization and cost reduction, has provided aerospace designers with powerful research tools for creat- 
ing new media for interactive, information displays. 

This flexibility raises many practical design challenges and interesting theoretical questions, 
but since many of these new information displays may be helmet or head mounted, particularly promi- 
nent questions concern guaranteeing the perceptual stability of the display's image. Indeed, it is argued 
in this paper that selecting a head-mounted format limits design freedom in the definition of the dis- 
plays in ways that do not constrain conventional panel-mounted formats. 

Analysis 

An understanding of the relevant design questions is best provided by an analysis of the linear 
transformations that the spatial information must undergo before presentation to the user. In general, 
the information is first defined as sets of vectors, polygons, or polyhedra positioned in an inertial ref- 
erence frame some times called the "real world" coordinate systems. (Foley and Van Dam, 1982). 

Prior to presentation to the viewer, this information must be transformed by scaling, rotation, 
translation, and projection to position it in an "eye coordinate system" determined by the position and 
direction of a viewing vector. This transformation processes is commonly represented as a series of ma- 
trix operations and is referred to as the "viewing transformation", but as shown in Figure 1, it may be 
broken into several separable parts each of which allows a unique opportunities for the introduction of 
informative distortions. 

Subsequent use of this spatial information by the viewer requires that he internally perform 
further coordinate transforms to bring it into a useful frame of reference. For example, if the subject is 
required to make an egocentric direction judgment based on information on a 3D map, he must further 


1 Earlier versions of this manuscript have been reported at the 1987 AGARD Meeting of the 
Aerospace Medicine Panel in Brussels, Belgium, September 28 - October 2, 1987 and at the 
1988 California Mapping Conference, San Jose, California. 
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transform the information into a body or even a hand centered coordinate system by a process similar to 
the viewing transformation. These are the transformations typically required in telerobotics. 




Figure 1. The process of representing a graphic object in the virtual space allows a number of different op- 
portunities to introduce informative geometric distortions or enhancements. These may either be a modification of 
the transforming matrix during the process of image definition or they may be modifications of an element of a 
model. Often the matrix element or shape of the model part is controlled externally by a variable slewed to a mouse 
ot other input device. These interventions may take place 1) in an object relative coordinate system used to define 
the object s shape or 2) in an affine or even curvilinear object transformation, or 3) during the placement trans- 
formation that positions the transformed object in world coordinates, or 4) in the viewing transformation. The per- 
ceptual consequences of informative distortions are different depending upon where they are introduced For ex- 
ample, object transformations will not impair perceptual stability in a head-mounted display whereas manipu- 
lations of the viewing transformation will. ¥ 7 F 
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In order to understand how the spatial information presented in pictures may be used, it is 
helpful to distinguish between images which may be described as spatial displays and those that were 
designed to be spatial instruments. One may think of a spatial display as any systematic mapping o 
one space onto another. A picture or a photograph is a spatial display. 


A spatial instrument, in contrast, is a spatial display that has been enhanced either by 
geometric or symbolic techniques to insure that the communicative intent of instrument is realized. A 
simple example of a spatial instrument is an analogue clock. In a clock the angular positions of the arms 
are made proportional to time, and the viewer s angle estimation task is assisted by radial tic marks 
designating the hours and minutes. A second aspect of the definition of a spatial instrument, which the 
clock example also illustrates, is that the communicated variable, time, is made proportional to a spa- 
tial property of the display, such as an angle, area, or length and is not simply encoded as a character 

string. 

The spatial instruments that we wish to focus attention on are generally interactive. That is to 
say that the communicated information flows both to and fro between the viewer and the instrument. 
Some of this bidirectional flow exists for practically all spatial instruments since movement of the 
viewer's viewpoint can have a major impact on the appearance of the display. However, the displays 
we wish to consider are those incorporating at least one controlled element, such as a cursor, which is 
used to extract information from and input information to the instrument. 

Maps also meet the definition of a spatial instrument. The map projection may be chosen de- 
pending upon the spatial property of importance. Choice of this projection illustrates objective geomet- 
ric enhancement. Overlaying of a graticule of latitude and longitude lines indicating the map metric is 
an example of symbolic enhancement. When fitted with these enhancements, the map can become a 
nomographic calculating instrument for navigation or spatial orientation. 


In selecting a map projection for a small scale map, such as a world map, a cartographer may 
select a projection from three families of perspective projections in which the solid angle formed at the 
point of contact between the globe and the projection surface varies 1) from 2n steradians, the zenithal 
case, 2) from 2 tc up to 0 steradians, the conical case, and 3) 0 steradians, the cylindrical case. 


Zenithal 
Solid angle * 2 n 


Conical 

Solid angle < 2a > 0 


Cylindrical 
Solid angle = 



Figure 3. Geometric categories of map projections. 

This selection can be guided by the ultimate use of the map. If, for example, a map is to be used 
to find minimum distance routes between distant points, a special case of the first type, the gnomonic 
projection can be used since it has the useful property of projecting all great circles as straight lines. 
(See Figure 4) One corresponding cost that must be incurred for this useful property is that the nonlinear 
scale distortion along meridians and parallels is unequal. Scale exaggeration for a piece of meridian in 
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the vicinity of latitude 6 is cosec^d and exaggeration of latitude scale is cosecB . The map is, thus, not 
orthomorphic, i.e. shape preserving (Cotter, 1966). Nonlinear scale distortions of this sort are, how- 
ever, well understood and can be objectively controlled by selection of the point of contact of the projec- 
tion and the extent of area represented on the map. 



Figure 4. Geometry of radial scale exaggeration in a gnomonic projection with an example of a gnomonic projection 
centered on the north pole. 


The distortions present on the gnomonic projection are a geometric consequence of the selected 
perspective parameters which describe the projection of the of the globe onto a tangent projection sur- 
face. They are not explicitly introduced but rather are a side-effect of the desired property that great 
circles map as straight lines. Distortion, however, can also be introduced directly into a projection to 
achieve a desired end as in popular conventional projections such as the mercator chart which is de- 
signed to map compass courses as straight lines. In this projection, a rectangular projection with the 
standard parallel set to the equator is intentionally distorted along the meridians to compensate for 
the fact that the scale along each parallel of the rectangular projection represents a smaller and 

smaller small circle as higher latitudes are mapped. Since the circumference, r, of a circle of latitude 6 

on a globe of radius R is: r = Rcos(9). each small element of the meridian must be stretched by l/cos(9) 
to compensate and straighten out the plots of oblique courses. The resulting scale exaggeration near the 
poles is so great to make the map unusable, but the technique works well for middle latitudes. 

The geometry of gnomonic or mercator projections is well understood and adapted to provide ge- 
ometric properties on the respective maps that are objectively useful. The straight line plotting of ei- 
ther great circles or rhumb lines facilitated use of the maps for navigation since desired courses could be 
found with a straight edge. Today in the time of computer-graphics based dynamic maps this advan- 
tage is not nearly as important. As is clearly evident from the software-based tools for interacting with 
spatial data bases as used for computer aided design, the old fashioned ruler and pencil have been 
generalized into multidimensional probes and almost magical cursors that can quickly extract highly 
dimensional spatial information from a data base which earlier generations of draftsman could barely 
imagine (Silicon Graphics, 1988; Dickinson, 1989ab). Consequently, the desired objective properties of 
images of spatial data are now not the same and in fact may be less important than the subjective ap- 
pearance of the space depicted in the image. It is this subjective appearance that most directly influ- 
ences the users interaction with the spatial data through his control of a cursor. Accordingly, the 
subjective appearance of the image now can become an important design feature. 

Spatial Instruments 

Contemporary spatial instruments are found throughout the modem aircraft cockpit, the most 
notable probably being the attitude direction indicator or ADI which displays a variety of signals re- 
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la ted to the aircraft's attitude and orientation with respect to earth based navigation aids. More re- 
cent versions of these standard cockpit instruments have been realized with CRT, cathode ray tubes, 
based instruments which have generally been modeled after their electromechanical predecessors 
(Boeing, 1983). 



y( (p) = j sec( 8) dd 
0 


Figure 5. Compensatory distortion of a rectangular projection is illustrated by the change in scale along the meridi- 
ans of the mercator projection. 

The computer graphics and CRT display media, however, allow the conception of totally novel 
display formats for demanding new aerospace applications. Grunwald and Ellis (Grunwald and Ellis, 
1988) have described, for instance, a more pictorial spatial instrument to assist informal, complex, or- 
bital navigation, proximity operations, and rendezvous in the vicinity of the space station (see Figure 
6). The definition of this instrument entailed a number of specific graphical enhancements which may 
be classified as either geometric, symbolic, or both. For example, a geometric enhancement was intro- 
duced by providing a display mode in which the axis along which spacecraft typically follow reen- 
trant looped paths is transformed into a time axis which does not exhibit these loops. This transforma- 
tion may assist obstacle avoidance and out of plane maneuvering during small orbital changes. The use 
of a time axis may also be a technique to avoid visual illusions associated with perspective projections 
of the trochoidal paths that describe the relative motion paths of one spacecraft with respect to each 
other. 



Figure 6. Sample proximity operations display. The solid curves lines show a planned orbital rendezvous between 
an orbital maneuvering vehicle (OMV) and the space station. The dotted line is a predicted flight path for the 
OMV. The projecting vectors show body axes of the craft. 
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Geometric Enhancement 


In general, there are various kinds of geometric enhancements that may be introduced into spa- 
tial displays, but their common feature is a transformation of the metrics of either the displayed space 
or of the objects it contains. A more familiar example is found in relief topographic maps for which it is 
useful to exaggerate the vertical scale. This technique has also been used for experimental traffic dis- 
plays for commercial aircraft. (Ellis, McGreevy, & Hitchcock, 1987) 

Another type of geometric enhancement important for displays of objects in 3D space involves 
the choice of the position and orientation of the eye coordinate system used to calculate the projection. 
Azimuth, elevation and roll of the system may be selected to project objects of interest with a useful as- 
pect. This selection is particularly important for displays without stereoscopic cues, but all types of 
displays can benefit from an appropriate selection of these parameters. (Ellis, Kim, Tyler, McGreevy 
and Stark, 1985; Kim, Ellis, Tyler, Hannaford, and Stark, 1987). 

Because of its dramatic effect on the image, selection of the field of view angle is particularly 
interesting. Only changing the field of view angle simply magnifies the image producing image which 
corresponds to an optic array geometrically similar to that optic array that a viewer would experience 
from the modeled eye point. Selecting a very wide field of view angle results in a minimized image, but 
also can introduce marginal distortions if a planar projection surface is used to produce the image. An 
additional source of distortion can arise if the display is viewed from a point other than the modeled 
eye point in the eye coordinate system. The effects of these latter distortions may, however, be modu- 
lated by the viewer’s awareness of the picture plane (Pirenne, 1970; Ellis, Smith, McGreevy, 1987). 

Significant design features can be achieved by joint variation of the field of view angle as ob- 
jects in the display (McGreevy and Ellis, 1986; Ellis, et al., 1987; Adams, 1975). Though this combined 
manipulation may introduce marginal distortions, it allows control over the projected sizes of objects in 
the image and, for example, allows definition of a projection that will always include a designated 
volume of the object space. This is a useful property of a situation awareness display which is not pre- 
served in a display by changes in the field of view alone. 

The introduction of deliberate spatial distortion into a spatial instrument can be a useful way to 
improve the communication of spatial information to a viewer since the distortion can be used to correct 
underlying natural biases in spatial judgements. For example, exocentric direction judgements (Howard, 
1982) made of extended objects in perspective displays, can for some response measures exhibit a 
"telephoto bias". That is to say that the subjects behave as if they were looking at the display through 
a telephoto lens. This bias can be corrected by introduction of a compensating wide-angle distortion. 
(McGreevy and Ellis, 1986; Grunwald and Ellis, 1987) 

Unnatural scaling by placement transformations can also be used to control an objects promi- 
nence, to insure, for example, that they never become vanishingly small, (see Figure 7). Scaling with 
an object transformation is also particularly effective at achieving nonlinear exaggerations but such un- 
natural object scaling can, however, increase display clutter: objects may interpenetrate. But indepen- 
dent scaling of the separate axes of the object generally provides the designer with techniques to reduce 
this interpenetration. 

Symbolic Enhancement 

Symbolic enhancements generally consist of objects, scales, or metrics that are introduced into a 
display to assist pick-up of the communicated information. The usefulness of such symbolic aids can be 
seen, for example, in displays to present air traffic situation information which focus attention on the 
relevant "variables" of a traffic encounter, such as relative altitude, as opposed to less useful 
"properties" of the aircraft state such as absolute altitude (Falzon, 1982). 

One way to present an aircraft's altitude relative to a pilot's own ship on a perspective display 
is to draw a grid at a fixed altitude below the "ownship" symbol and drop reference lines from all air- 


526 



craft symbols onto the grid. If the "ownship" altitude is marked on these reference lines then the dis- 
tance from the other aircraft symbol to the mark is proportional to the relative altitude. If the aircraft 
are given predictor vectors that show future position, similar reference lines can be dropped from the 
ends of the predictor lines. 

The reference lines not only serve to clarify the target's ambiguous aspect but also can improve 
perception of the target s heading. This effect has been shown in a recent experiment examining the ef- 
fects of reference lines on egocentric perception of azimuth of extended objects in perspective images cre- 
ated by a microcomputer graphics system. This experiment provides a specific example of how psy- 
chophysical evaluation of display formats can be used to assess their information display effective- 
ness. 


In this experiment 10 subjects viewed static perspective projections of aircraft-like symbols ele- 
vated at three different levels above a ground reference grid: a low level below the view vector and 
almost on the grid, a middle level co-linear with the viewing vector, and a high level above the view 
vector by the same amount as the low level was below it (see Figure ). The aircraft symbols have 
straight predictor vectors projecting forward showing future position above the reference grid. In one 
condition reference lines were dropped only from the current aircraft position. In the second condition 
reference lines were dropped also from the ends of lines projecting from each aircraft. These lines could 
represent predictors of future position. 


The subjects viewed the entire configuration of aircraft symbol and grid from a fixed eye posi- 
tion 28 cm from the projection surface. This position was at the geometricaly correct center of projection 
for a viewing vector set to 0 degrees azimuth and -22.5 deg elevation. Nine different azimuth rotations 
of the image were presented: 0 to 180 in 22.5 degree increments. The subject's task was to adjust the ego- 
centric direction of a horizontal dial to indicate the azimuth rotation of the aircraft. Azimuth rotation 
was crossed with number of reference lines in a factorial repeated measures experiment. 



Figure 7. Sample cockpit display of air traffic. Own ship is at the center of the display. 1 minute predictors project 
out of all aircraft symbols. Reference line are dropped perpendicular to the reference grid. 


The first result of the experiment was that subjects made a substantial errors in their estimation 
of the azimuth rotation of the aircraft; they generally saw it rotated more towards the picture plane 
than it in fact was.(F= 23.4, df = 8,72; p. < .001) This corresponded to clockwise errors for actual clock- 
wise rotations up to 90 degrees. The errors reverse for rotations greater than 90 degrees. 
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Figure 8. Five views of sample stimuli used which illustrate the three heights of the aircraft symbol above the grid 
and the two reference line conditions. Viewing elevation = -22.5 deg, azimuth = 45 degrees. As the aircraft rises 
away from the grid it develops an illusory yaw toward the picture plane. 

The second result is that the the error towards the frontal plane for the symbols with one 
reference line increased as the height of the symbol increased above the grid (F= 4.1, df ■ 2,18, p < .34). 
Most significantly, however, as shown in Figure 9, introduction of the second reference line totally 
eliminated the effect of height, reducing the azimuth error in some cases about 50% (F = 2.402, df = 
16,144, p < .003). A more detailed geometric and perceptual analysis of this result is beyond the scope 
of this paper; however, these experimental results show in a concrete way how appropriately chosen 
symbolic enhancements can provide not only qualitative but quantitative improvement in pictorial 
communication. 



Figure 9. Mean clockwise and counterclockwise egocentric direction judgement for clockwise azimuth rotation. 

Combined Geometric and Symbolic Enhancements 

Some enhancements combine both symbolic and geometric elements. One good example is pro- 
vided by techniques connecting the photometric properties of objects or regions in the display with 
other geometric properties of the objects or regions themselves. Russell and Miles (1987), for example. 
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have associated the optical density of points in space with the norm of the gradient of the concentra- 
tion of a dissolved component and produced striking visualization of three-dimensional distributions of 
the compound. Similar techniques have been applied to solid models derived from sequences of CAi 
scans and allowed a kind of "electronic dissection" of medical images by control of the transparency of 
the different tissue types contained in the X-ray images (Meagher, 1987). Though this technique can 
provide absolutely remarkable images; one could for example "see the wind by making optical density 
proportional to velocity; one of the challenges of its use is the introduction of metrical aids to allow the 
viewer to pickup quantitative information from the photometric transformation. 


nicnicfiirm 


The different types of enhancement are important in particular for head-mounted displays be- 
cause they interact differently with the image and viewer. The global geometric enhancements are 
particularly important for head-mounted displays since they interfere with visual-vestibular 
coordination and can result in motion sickness. 

Computer generated, helmet-mounted images were probably first produced by Ivan Sutherland 
in 1970 (Sutherland, 1970) and have more recently been produced somewhat more elaborately at several 
other laboratories. (Furness, 1986; Fisher, McGreevy, Humphries, Robinett, 1986). When Sutherland 
developed his display, the required hardware and software investment was substantial and available 
only to well funded laboratories. In contrast today, the display technology has become so inexpensive 
that a system adequate for creditable research can be assembled within a budget of a few thousand 
dollars. 


Presentation of the computer generated image display on a head mounted display strongly en- 
courages the viewer to interpret the projection as a virtual space which is expected to interact with his 
movements as if it were a real space. This kind of interpretation also occurs, but to a lesser extent, with 
ordinary pictures presented in the normal panel mounted format. The interpretation of a virtual space 
can give rise to pictorial illusions of depicted orientation (Goldstein, 1987; Ellis, Smith, and McGreevy, 
1987), but these effects are far weaker with panel mounted displays than with those that are helmet- 
mounted. 

One reason for the difference is that the helmet displays often include collimating optics, 
(Weintraub et al., 1985) producing true virtual images and interfering with viewers ability to locate 
the surface of the picture (Nagata, 1986). Furthermore, the helmet displays generally present wider 
fields than the panel mounted displays. These viewing conditions, which trigger the normal binocular 
reflexes associated with vergence accommodation, coupled with the vestibular effects of head move- 
ment result in a viewing situation that requires careful calibration to insure perceptual stability. If 
stereoscopic presentation or head driven motion parallax are used, this requirement is assured. 

The difficulty with this format is that the global geometric enhancements destroy the required 
calibration. This difficulty is true by definition for the enhancements, such as differential scaling of 
the display axes, that operate on the viewing transformation itself, but it is also true, though to a 
lesser extent, of enhancements such as differential object scaling because familiar size can be the over- 
riding cue to apparent distance (Ittelson, 1951). This effect may have operational significance and ex- 
plain errors pilots make when using virtual image displays (Roscoe, 1984; 1987). 

The loss of visual stability due to improper correlation between visual and vestibular move- 
ment arises from both voluntary and involuntary head movement. Large voluntary head movements can 
produce the most obvious loss of stability if the gains and phase lags between the image movement and 
vestibular ocular reflex (VOR) do not match. Fortunately, the VOR is adaptable and can adjust its gam 
and phase response (Bertoz and Melville-Jones 1985), though time lags resembling transport delays 
may preclude this adaptation. Small involuntary head movements cause relative movement between 
the head and the viewing axis of the eye which is inertially stabilized by the VOR. In this situation 
the head-mounted display screen moves and blurs the image. Thus the normal operation of the VOR is 
actually counterproductive. Measurement of the actual head movement can provide a signal to allow 
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compensatory, inertial stabilization of the display by displacement on the screen by adaptive filters 
which can model the VOR (Wells and Griffin, 1984; Velger et al, 1988). 

Besides loss of visual stability, geometric enhancements can interfere with visuo-motor coordi- 
nation. This interference is particularly evident if the display includes a hand-controlled cursor. Under 
these circumstances an improperly calibrated or and intentionally distorted display resembles the 
view through a prism and lens system that introduces an optical distortion into the lines of sight. As 
known at least from the time of Helmholtz (1856), the visuo-motor system can completely adapt to the 
kind of conformal transformation such system can produce. Short time delays, on the order of 100 msec., 
can, however, substantially degrade or block this adaptation. (Held, Efstathiou, and Greene, 1966). 

Allowable Enhancements for Helmet Mounted Instruments 

In view of the many intrinsic problems with purely geometrical enhancement, the safest en- 
hancements for helmet mounted instruments seem to be symbolic, the kind of added information over- 
lays that have been used on aircraft head-up-displays for years. 

These displays typically transpose much of the information already available in aircraft 
cockpits into a more integrated form and present it on a large combining plate, or beam splitter, so the 
information is available "head up" and can be seen when the pilot looks out the window (Weintaub, 
Haines and Randle, 1985). In addition to the usual moving tape, cursors, or numerical readouts, these 
displays often have a small graphics image projected to correspond in shape, size and position to an 
out-the-window object such as a runway. Maintaining good calibration for such an overlap between a 
display-generated graphics object and the projection of a real external object represents a significant 
challenge in a wearable helmet not using skull screws to maintain its position on the users head. In- 
deed, helmet mounted displays of this sort have been suggested as a useful nausea-inducing apparatus 
to attempt to habituate astronauts to the sensory discordance of weightlessness before they begin space 
travel. (Parker, Renschke, Arrott, Homick, and Lichtenberg, 1986). 

Never the less, symbolic use of three-dimensions also seems to be an allowable enhancement. 
For example, one can imagine three-dimensional icons representing records in a hierarchical data base 
for which the third dimension could represent depth of nesting. Another interesting possibility for 
symbolic aid could be transient 3D "yardsticks" used in combination with a 3D cursor to designate pairs 
of objects to be compared. Once two objects are selected, a line symbolically designating their separation 
could be temporarily generated to display a binary relation between them. 

Among the geometric enhancement, those least likely to cause visual stability problems are 
those that act on the real world coordinates of the displayed objects themselves: the object scaling 
transformations. Provided that the transformed objects do not markedly violate the viewers implicit 
assumptions about size and shape, these transformations act early enough so that their effect may in- 
terpreted simply as changing the shape and size of the objects. They would unfortunately interfere 
with manual manipulation of the objects, but as long as this is carried out symbolically with a cursor 
and not with a simulated "hand” with many degrees of control which must be adapted to the conditions 
of the display space, these size and shape transformation should not be too aversive. 

Finally, the photometric transformation illustrated by Russell and Miles (1987) is unlikely to 
have untoward consequences for head mounted instruments and may prove useful if combined with met- 
rical aids allowing them to present more quantitative information. 

In the final analysis the basic limits in the definition of helmet mounted instruments may not 
be classically technological, but intellectual. The technological limits faced in the design of these tools 
will be foreseeably over come by time and effort of others involved with the seemingly inevitable 
progress of optical and electronic fabrication. The intellectual limits will be overcome only by designers 
imagination and understanding of human spatial perception. 
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